Skip to content

feat(c): add bounded commit-message range export - #1330

Merged
JingsongLi merged 1 commit into
apache:mainfrom
suxiaogang223:codex/bounded-commit-export
Nov 5, 2026
Merged

JingsongLi merged 1 commit into
apache:mainfrom
suxiaogang223:codex/bounded-commit-export

Conversation

@suxiaogang223

Copy link
Copy Markdown
Member

Purpose

Linked issue: Closes #1327.

Related to #910 and #1257.

An embedding engine needs to export bounded ranges before passing commit bodies to a size-limited
RPC. The existing API serializes every message before a caller can check the
resulting size.

Prepared with AI assistance.

Brief change log

  • Add CommitMessage::serialize_with_limit(max_bytes), checking before each
    output append while preserving the existing serializer and Java v14 bytes.
  • Add paimon_commit_messages_count(messages, count_output) and
    paimon_commit_messages_serialize_range(messages, offset, count, max_bytes).
  • Reuse the existing byte-array result and free function. Validate range
    overflow and bounds; return no partial output on failure and preserve the
    source handle for a smaller-range retry.

Tests

Validated with the existing Rust 1.97.0 toolchain on macOS arm64:

  • cargo +1.97.0 test --locked -p paimon-c --lib: 95 passed, including the existing
    separate worker/coordinator process test.
  • New coverage checks Java fixture equality, exact limits, one-byte-smaller
    limits, cumulative limits, nonzero offsets, empty/invalid/overflowing ranges,
    null handles and output pointers, retry, output lifetime and Format Table
    rejection.
  • cargo +1.97.0 test --locked -p paimon --lib table::commit_message::tests: 3 passed.
  • cargo +1.97.0 fmt --all -- --check and git diff --check: passed.
  • cargo +1.97.0 clippy --locked -p paimon-c --all-targets -- -D warnings: passed.

Not run locally: CI-pinned Rust 1.98.0 checks, the full workspace/Vortex
suite, or the Linux/MSRV matrix. The results above cover the listed commands.

API and Format

Additive Rust and C APIs. Existing full export and Java v14 body bytes remain
unchanged. No Java list wrapper, storage format or engine-specific framing is
added. The limit bounds the sum of body lengths, not peak allocations or RSS:
temporary metadata encoding, vector capacity and ownership conversion remain.

Documentation

Add a range-export/retry example and explain byte limits and ownership in the
C integration guide.

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@JingsongLi
JingsongLi merged commit 856cce2 into apache:main Nov 5, 2026
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add bounded range export for C commit-message bodies

2 participants